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ABSTRACT 



In 1990, the National Assessment of Educational Progress 
(NAEP) included a Trial State Assessment (TSA) ; for the first time in NAEP's 
history, voluntary state-by-state assessments were made. In 1994, TSA was 
expanded to include non-public school students. The 1994 reading assessment 
considered students' proficiency in situations that involved reading 
different kinds of materials for different purposes. The fourth-grade 
assessment measured two global purposes for reading- -reading for literary 
experience and reading to gain information. In Kentucky, 2,758 students in 
101 public schools and 278 students in 12 non-public schools were assessed. 
This report describes the reading proficiency of Kentucky fourth-graders and 
compares their overall performance to students in the Southeast region of the 
United States and the nation (using data from the NAEP national assessments) . 
The distribution of reading results and reading achievement level results are 
provided for subpopulations of students (race/ethnicity, type of location, 
parents, educational level, and gender) . Reading proficiency and achievement 
of non-public school students are separately reported and compared to public 
school students. To provide a context for the assessment data, participating 
public school students, their reading teachers, and principals completed 
questionnaires which focused on: policies and practices related to reading 
(time for instructional activities and instructional resources for reading) ; 
delivery of reading instruction (instructional materials and activities, 
workbooks/ worksheets/writing, discussion and group activities, time to read, 
reading and use of libraries, assessing progress in reading) ; reading 
instructors (preparation, experience and professional development); students' 
home support for literacy (reading outside of school and in the home and 
hours of television watched per day) . The average reading proficiency of 
fourth-grade public school students in Kentucky on the NAEP reading scale was 
212 compared to 212 nationwide. (SR) 



erJc 




(MSP 'DPP® BSMDOOO® Qmm BSP®Btf 

(EGGOtfOOT 





the mom 

ERIC 



Points of view or opinions stated in this 
document do not necessarily represent 
MiciflLQERI position or. policy. 



0Do§o BEPflBTOfflliOT @1? IIBXKOTOK) 

®p[?0(gii ®f? n®QD€a'u“o®raaa besbaot m® OGaTOWEEMiJ 






What is The Nation’s Report Card? 



THE NATION’S REPORT CARD, the National Assessment of Educational Progress (NAEP), is the only nationally representative and continuing 
assessment of what America’s students know and can do in various subject areas. Since 1969, assessments have been conducted periodically in 
reading, mathematics, science, writing, history/geography, and other fields. By making objective information on student performance available to 
policymakers at the national, state, and local levels, NAEP is an integral part of our nation’s evaluation of the condition and progress of education. 
Only information related to academic achievement is collected under this program. NAEP guarantees the privacy of individual students and their 
families. 

NAEP is a congressionally mandated project of the National Center for Education Statistics, the U.S. Department of Education. The 
Commissioner of Education Statistics is responsible, by law, for carrying out the NAEP project through competitive awards to qualified organiza- 
tions. NAEP reports directly to the Commissioner, who is also responsible for providing continuing reviews, including validation studies and 

solicitation of public comment, on NAEP’s conduct and usefulness. 

In 1988, Congress established the National Assessment Governing Board (NAGB) to formulate policy guidelines for NAEP. The Board is 
responsible for selecting the subject areas to be assessed from among those included in the National Education Goals; for setting appropriate student 
performance levels; for developing assessment objectives and test specifications through a national consensus approach; for designing the 
assessment methodology; for developing guidelines for reporting and disseminating NAEP results; for developing standards and procedures for 
interstate, regional, and national comparisons; for determining the appropriateness of test items and ensuring they are free from bias; and for taking 
actions to improve the form and use of the National Assessment. 
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EXECUTIVE 

SUMMARY 



In 1988, Congress passed legislation for the National Assessment of Educational 
Progress (NAEP) that continued its primary mission of providing dependable and 
comprehensive information about educational progress in the United States. In addition, 
for the first time in the project’s history, the legislation also included a provision 
authorizing voluntary state-by-state assessments on a trial basis. 

As a result of the legislation, the 1990 NAEP program included a Trial State 
Assessment Program in which public-school students in 37 states, the District of 
Columbia, and two territories were assessed in eighth-grade mathematics. 1 The 1992 
NAEP program included an expanded Trial State Assessment Program in fourth-grade 
reading and fourth- and eighth-grade mathematics, with public-school students assessed 
in 41 states, the District of Columbia, and two territories. 2 

The continuation of NAEP’s Trial State Assessment Program in 1994 was 
authorized by additional legislation that enlarged the state-by- state assessment to include 
non-public school students. In addition to the state assessment program in reading at 
grade 4, the 1994 NAEP involved national assessments of reading, world geography, 
and U.S. history at grades 4, 8, and 12. The 1994 Trial State Assessment Program was 
conducted in February 1994 with 44 participants (41 states, the District of Columbia, 
Guam, and the Department of Defense Education Activity [DoDEA] Overseas Schools). 

This computer-generated report describes the reading proficiency of fourth-grade 
students in Kentucky, the Southeast region, and the nation. The distribution of reading 
proficiency results and reading achievement level results are provided for groups of 
students defined by shared characteristics: race/ethnicity, type of location, parents’ 
education level, and gender. Contextual information about reading policies, instruction, 
and home support for reading is presented for public school students. State results are 
based on the representative sample of students who participated in the 1994 Trial State 
Reading Assessment Program. Results for the region and the nation are based on the 
regional and national representative samples of students who participated in the national 
NAEP assessmen t. 

1 For a summary of the 1990 program, see Ina V.S. Mullis, John A. Dossey, Eugene H. Owen, and Gary W. Phillips. 
The State of Mathematics Achievement: NAEP’s 1990 Assessment of the Nation and the Trial Assessment of the States. 
(Washington, DC: National Center for Education Statistics, 1991). 

2 

For a summary of the 1992 assessment of reading, see Ina V.S. Mullis, Jay R. Campbell, and Alan E. Farstrup. The 
NAEP 1992 Reading Report Card for the Nation and the States. (Washington, DC: National Center for Education 
Statistics, 1993). For a summary of the 1992 assessment of mathematics, see Ina V.S. Mullis, John A. Dossey, Eugene 
H. Owen, and Gary W. Phillips. NAEP 1992 Mathematics Report Card for the Nation and the States. (Washington, 
DC: National Center for Education Statistics, 1993). 
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School and Student Participation in the Reading Assessment 

In Kentucky, 101 public schools and 12 non-public schools participated in the 
1994 fourth-grade reading assessment. These numbers include participating substitute 
schools that were selected to replace some of the nonparticipating schools from the 
original sample. The weighted school participation rate after substitution in 1994 was 

96 percent for public schools and 85 percent for non-public schools, which means that 
the fourth-grade students in this sample were directly representative of 96 percent and 
85 percent of all the fourth-grade public and non-public school students in Kentucky, 
respectively. 

In Kentucky, 2,758 public school and 278 non-public school fourth-grade students 
were assessed in 1994. The weighted student participation rate was 97 percent for 
public schools and 97 percent for non-public schools. This means that the sample of 
fourth-grade students who took part in the assessment was directly representative of 

97 percent of the eligible public school student population and 97 percent of the 
eligible non-public school student population in participating schools in Kentucky (that 
is, all students from the population represented by the participating schools, minus those 
students excluded from the assessment). 

The overall weighted response rate (school rate times student rate) was 93 percent 
and 82 percent for public and non-public schools, respectively. This means that the 
sample of students who participated in the assessment was directly representative of 
93 percent of the eligible fourth-grade public school population and 82 percent of the 
eligible fourth-grade non-public school population in Kentucky. 

Following standard practice in survey research, the results presented in this report 
were produced using calculations which incorporate adjustments for the nonparticipating 
schools and students. Hence, the final results derived from the sample provide estimates 
of the reading proficiency and achievement for the full population of eligible public and 
non- public school fourth-grade students in Kentucky. However, these nonparticipation 
adjustments may not adequately compensate for the missing sample schools and students 
in instances where nonparticipation rates are large. 

In order to guard against potential nonparticipation bias in published results, NCES 
has established minimum participation levels necessary for the publication of 1994 Trial 
State Assessment results. NCES also established additional guidelines that address four 
ways in which nonparticipation bias could be introduced into a jurisdiction’s published 
results (see Appendix A). In 1994, Kentucky met minimum participation levels for 
public and non-public schools. Hence, results for both types of schools are included in 
this report. Kentucky met all other established NCES participation guidelines for public 
schools but failed to meet one or more of these guidelines for non-public schools. 
Kentucky’s weighted participation rate for the initial samples of non-public schools was 
below 85 percent and the weighted school participation rate after substitution was below 
90 percent (see Appendix A). 
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Students’ Reading Performance 

The table below shows the distribution of reading proficiency of fourth-grade 
students attending public schools in Kentucky, the Southeast region, and the nation. 

1994, Public School Students 

The average reading proficiency of fourth-grade public school students 
in Kentucky on the NAEP reading scale was 212. This average was not 
significantly different from that of students across the nation (21 2). 3 
The lowest performing 10 percent of public school fourth graders in 
Kentucky had proficiencies at or below 161 while the top 10 percent 
had proficiencies at or above 259. In public schools across the nation, 
the lowest performing 10 percent of fourth graders had proficiencies at 
or below 156; the top performing 10 percent of students had 
proficiencies at or above 261. 

1992 vs 1994, Public School Students 
There was no significant change in the average performance of 
fourth-grade public school students in Kentucky from 1992 to 1994 (213 
in 1992 and 212 in 1994). During the same period, there was no 
significant change in the average performance of fourth-grade public 
school students across the nation (215 in 1992 and 212 in 1994). 
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The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). If the notation > (<) appears, it signifies that the value for public school students 
in 1994 was significantly higher (lower) than the value for 1992 at about the 95 percent confidence level. 



Differences reported as significant are statistically different at the 95 percent confidence level. This means that with 
95 percent confidence there is a real difference in the average reading proficiency between the two populations of 
interest. 
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Performance According to Purpose for Reading 

The 1994 Trial State Assessment Program considered students’ performance in 
situations that involved reading different kinds of materials for different purposes. The 
fourth-grade reading assessment measured two global purposes for reading — reading 
for literary experience and reading to gain information. The table below provides 
results for Kentucky, the Southeast region, and the nation according to each reading 
purpose. 



1994, Public School Students 

The proficiency of public school students in Kentucky in reading for 
literary experience (213) was not significantly different from that of 
students across the nation (214). Similarly, in reading to gain 
information, the proficiency of public school students in Kentucky (210) 
did not differ significantly from that of students across the nation (210). 

1992 vs 1994, Public School Students 
Kentucky’s public school fourth graders showed no significant change 
from 1992 to 1994 in reading for literary experience. Similarly, in 
reading to gain information, public school fourth graders in Kentucky 
exhibited no significant change from 1992 to 1994. 
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The NAEP “purpose for reading” scales range from 0 to 500. The standard errors of the statistics appear in parentheses. 
It can be said with about 95 percent confidence that, for each population of interest, the value for the entire population 
is within ± 2 standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error 
of the difference (see Appendix A for details). If the notation > (<) appears, it signifies that the value for public school 
students in 1994 was significantly higher (lower) than the value for 1992 at about the 95 percent confidence level. 
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Levels of Reading Achievement 

The most recent reauthorization of the National Assessment Governing Board 
(NAGB) continues the Board’s responsibilities to set policy for NAEP and to “develop 
appropriate student achievement levels for each age and grade in subject areas tested” 
(Pub. L. 103-382). 

NAGB developed three achievement levels for each grade — Basic, Proficient, 
and Advanced. Performance at the Basic level denotes partial mastery of the knowledge 
and skills that are fundamental for proficient work at each grade level. The central level, 
called Proficient, represents solid academic performance at each grade level tested. 
Students reaching this level demonstrate competency over challenging subject matter and 
are well prepared for the next level of schooling. Performance at the Advanced level 
signifies superior performance at the grade tested. Definitions of the three levels of 
reading achievement are given below. Chapter 3 provides further elaboration of these 
levels and presents examples of types of questions that students at each of the three 
achievement levels can respond to effectively. 
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Description of Fourth-Grade Reading Achievement 
Levels 



Achievement 

Level 


Scale 
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ADVANCED 
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Fourth-grade students performing at the Advanced /eve/ 
should be able to generalize about topics in the reading 
selection and demonstrate an awareness of how authors 
compose and use literary devices. When reading text 
appropriate to fourth grade, they should be able to judge 
texts critically and, in general, give thorough answers that 
indicate careful thought. 
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Fourth-grade students performing at the Proficient level 
should be able to demonstrate an overall understanding 
of the text, providing inferential as well as literal 
information. When reading text appropriate to fourth grade, 
they should be able to extend the ideas in the text by 
making inferences, drawing conclusions, and making 
connections to their own experiences. The connection 
between the text and what the student infers should be 
clear. 


iiiiiiifii 

; .> -.i 
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Fourth-grade students performing at the Basic level should 
demonstrate an understanding of the overall meaning of 
what they read. When reading texts appropriate for fourth 
graders, they should be able to make relatively obvious 
connections between the text and their own experiences. 
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The table below provides the percentage of fourth-grade students at or above each 
achievement level, as well as the percentage of students below the Basic level. 

1994, Public School Students 

The percentage of public school students in Kentucky who were at or 
above the Proficient level (26 percent) did not differ significantly from 
that of students across the nation (28 percent). 

1992 vs 1994, Public School Students 
From 1992 to 1994, there was no significant change in the percentage 
of public school students in Kentucky who attained the Proficient level 
(23 percent in 1992 and 26 percent in 1994). Similarly, there was no 
significant change in the percentage of public school students across the 
nation who attained the Proficient level (27 percent in 1992 and 
28 percent in 1994). 
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Levels of Fourth-Grade Public School Students’ 
Reading Achievement 



At or Above 



At or Above 



At or Above 
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1994 Trial Stale Assessment 



Advanced 
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1992 Public 

Kentucky 
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Nation 
1994 Public 
Kentucky 
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The standard errors of the statistics appear in parentheses. It can be said with about 95 percent confidence that, for each 
population of interest, the value for the entire population is within ± 2 standard errors of the estimate for the sample. In 
comparing two estimates, one must use the standard error of the difference (see Appendix A for details). If the notation 
> (<) appears, it signifies that the value for public school students in 1994 was significantly higher (lower) than the value 
for 1992 at about the 95 percent confidence level. 



Subpopulation Performance 

Assessment results repeatedly show differences in performance for subpopulations 
of students. 4 The 1994 Trial State Assessment provides additional information about 
the performance of important subpopulations by reporting on the reading proficiencies 
of various subgroups of the public school student population defined by race/ethnicity, 
type of location, parents’ education level, and gender. These results are summarized in 
the table on page 8. 
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Ina V.S. Mullis, John A. Dossey, Jay R. Campbell, Claudia A. Gentile, Christine O’Sullivan, and Andrew S. Latham. 
NAEP 1992 Trends in Academic Progress . (Washington, DC: National Center for Education Statistics, 1994). 
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Race/Ethnicity 

1994, Public School Students . The average reading proficiency of White 
students in Kentucky public schools was higher than that of Black and 
Hispanic students. 

7992 vs 1994 , Public School Students. There was no significant change 
in the average reading proficiency of White, Black, or Hispanic public 
school students in Kentucky from 1992 to 1994. 

Type of Location 

1994 , Public School Students. The average reading proficiency of 
Kentucky students attending public schools in central cities was not 
significantly different from that of students in urban fringe/large towns 
and rural areas/small towns. 

1992 vs 1994 , Public School Students. From 1992 to 1994, there was 
no significant change in the average reading proficiency of students 
attending public schools in central cities, urban fringe/large towns, or 
rural areas/small towns in Kentucky. 

Parents 9 Education Level 

1994 , Public School Students. Public school students in Kentucky 
reporting that at least one parent graduated from college demonstrated 
an average reading proficiency which did not differ significantly from 
that of students who reported that at least one parent had some education 
after high school or at least one parent graduated from high school but 
was higher than that of students who reported that neither parent 
graduated from high school or they did not know their parents’ 
education level. 

7992 vs 1994 , Public School Students. The average proficiency of public 
school students in Kentucky who reported that at least one parent 
graduated from college, at least one parent had some education after 
high school, at least one parent graduated from high school, neither 
parent graduated from high school, or they did not know their parents’ 
education level did not change significantly between 1992 and 1994. 

Gender 

1994 , Public School Students. In public schools in Kentucky, girls 
exhibited an average reading proficiency which was higher than that of 
boys. 

7992 vs 1994 , Public School Students. In Kentucky public schools, the 
average reading proficiency for boys did not change significantly from 
1992 to 1994. Similarly, the average proficiency for girls did not change 
significantly from 1992 to 1994. 
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* School sample size is insufficient to permit reliable regional results for type of location. 

The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). If the notation > (<) appears, it signifies that the value for public school students 
in 1994 was significantly higher (lower) than the value for 1992 at about the 95 percent confidence level. ! Interpret 
with caution — the nature of the sample does not allow accurate determination of the variability of this statistic. 
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Reading Performance of Non-Public School Students 

The 1994 Trial State Assessment marks the first time that non-public school 
students were assessed at the state level. Therefore, separate non-public school results 
can be reported for Kentucky. Also, results based on a combined sample of public and 
non-public school students can be presented. The following table shows the distribution 
of overall reading proficiency for the non-public school and combined populations. The 
corresponding public school student results can be found in the table on page 3. 

Non-Public School Students 

The average reading proficiency of fourth-grade students in non-public 
schools in Kentucky was 237. This average was not significantly 
different from that of non-public school students across the nation (231). 

Public and Non-Public School Student Comparison 
The average reading proficiency of fourth-grade students in non-public 
schools in Kentucky was higher than the average for public school 
students (237 for non-public and 212 for public). For the nation, the 
average reading proficiency for non-public school students was higher 
than that of their public school counterparts (231 for non-public and 212 
for public). 

1994, Public and Non-Public School Students Combined 
The average reading proficiency of grade 4 students from Kentucky was 
213. This average did not differ significantly from that of students 
across the nation (214). The lowest performing 10 percent of fourth 
graders from Kentucky had proficiencies at or below 162 while the top 
10 percent had proficiencies at or above 261. The lowest performing 
10 percent of students across the nation had proficiencies at or below 
159 while the top 10 percent had proficiencies at or above 263. 
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The NAEP reading scale ranges from 0 to 500. The standard errors of the statistics appear in parentheses. It can be said 
with about 95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 
standard errors of the estimate for the sample. In comparing two estimates, one must use the standard error of the 
difference (see Appendix A for details). 
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A Context for Understanding Students’ Reading Proficiency in Public Schools 

Information on the reading performance of students in Kentucky can be better 
understood and used for improving instruction and setting policy when supplemented 
with contextual information about schools, teachers, and students. 

To gather contextual information, the fourth-grade students participating in the 
1994 Trial State Assessment, their reading teachers, and the principals or other 
administrators in their schools were asked to complete questionnaires on policies, 
instruction, and programs. The student, teacher, and school data help to describe some 
of the current practices and emphases in reading education, illuminate some of the 
factors that appear to be related to fourth-grade public-school students’ reading 
proficiency, and provide an educational context for understanding information on student 
achievement. Highlights of the results for the public-school students in Kentucky are 
as follows: 

CURRICULUM COVERAGE AND INSTRUCTIONAL EMPHASIS 

• In Kentucky in 1994, average reading proficiency was similar for 
students regardless of how much time their reading teachers spent on 
reading instruction on a typical day. 

• According to the public school administrators in Kentucky, in 1994, 

85 percent of the fourth-grade students were in schools where reading 
was identified as receiving special emphasis. This percentage was not 
significantly different from that of students across the country 
(85 percent). 

• In 1994, according to their reading teachers, 13 percent of the students 
in public schools in Kentucky were typically taught reading in a class 
that was grouped by reading ability. The prevalence of ability grouping 
was higher across the nation (22 percent). 



DELIVERY OF READING INSTRUCTION 

• Students in Kentucky whose teachers used both basal and trade books 
demonstrated an average reading proficiency (212) which did not differ 
significantly from that of students whose teachers primarily used basal 
readers (213). 

• The proficiency of Kentucky students whose teachers used both basal 
and trade books (212) was not significantly different from that of 
students whose teachers primarily used trade books (211). 

• The proficiency of Kentucky students whose teachers primarily used 
trade books (211) was not significantly different from that of students 
whose teachers primarily used basal readers (213). 
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• In Kentucky, 42 percent of the fourth-grade students had reading 
teachers who used children’s newspapers and/or magazines at least once 
a week; 16 percent of the students had reading teachers who used 
reading kits at least once a week; 36 percent had reading teachers who 
used computer software for reading instruction at least once a week; 
83 percent of the students had reading teachers who used a variety of 
books at least once a week; and, finally, 74 percent of the students had 
reading teachers who used materials from other subject areas at least 
once a week. 

• According to the Kentucky reading teachers, 63 percent of the students 
were asked to discuss new or difficult vocabulary almost every day. 
This percentage did not differ significantly from that of students across 
the nation (62 percent). 

• According to their reading teachers, the percentage of students in 
Kentucky who were asked to talk with each other almost every day 
about what they have read (32 percent) was not significantly different 
from that of students across the nation, where 34 percent of the students 
were asked to do this activity almost every day. 

• According to the reading teachers in Kentucky, 3 percent of the students 
were asked to do a group activity or project about what they have read 
almost every day. This figure was not significantly different from that 
of students across the nation (5 percent). 



EDUCATIONAL BACKGROUND OF TEACHERS 

• The percentage of students who were being taught by reading teachers 
who reported having at least a master’s or education specialist’s degree 
in Kentucky (82 percent) was greater than that for the nation 
(41 percent). 

• About two thirds of the students (69 percent) had reading teachers who 
had the highest level of teaching certification that is recognized by 
Kentucky. This did not differ significantly from the figure for the 
nation, where 65 percent of the students were taught by reading teachers 
who were certified at the highest level available in their states. 

• In Kentucky, 14 percent of the students were being taught reading by 
teachers who had an undergraduate major in English, reading, and/or 
language arts. This was smaller than the percentage of students across 
the nation who were being taught by reading teachers with the same 
major (20 percent). 
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HOME FACTORS 

• In Kentucky, 32 percent of the students reported having four types of 
reading materials (a newspaper, an encyclopedia, 25 or more books, and 
magazines) in the home. This figure was somewhat lower than that for 
the nation (36 percent). Students in Kentucky who had all four of these 
types of materials in the home showed an average reading proficiency 
(222) which was higher than that of students with zero to two types of 
materials (199). 

• In 1994 in Kentucky, 28 percent of the students discussed with friends 
or family what they read almost every day. This percentage did not 
differ significantly from that of students across the nation (28 percent). 

The proficiency of students in Kentucky who discussed what they read 
with friends or family almost every day (211) did not differ significantly 
from that of students who had discussions with friends or family less 
than weekly (207). 

• About one quarter of the fourth-grade students (25 percent) watched six 
hours or more of television each day. This was somewhat larger than 
the figure for the nation, where 22 percent of the students watched this 
much television. Average reading proficiency in Kentucky was lowest 
for students who spent six hours or more watching television each day. 

Comparisons of Overall Reading Proficiency in Kentucky with Other States 

The map on the following page provides a method for making appropriate 
comparisons of the overall public school reading proficiency in Kentucky with that in 
other states (including Guam and the Department of Defense Education Activity 
[DoDEA] Overseas Schools) that participated in the NAEP 1994 Trial State Assessment 
Program. The different shadings of the states on the map show whether the average 
overall proficiency of public school students in the other states was statistically different 
from or not statistically different from that of public school students in Kentucky 
(“Target State”). States in black have a significantly lower average public school 
proficiency than does Kentucky. States with a dark-gray shading have a significantly 
higher average public school proficiency than does Kentucky. States with a light-gray 
shading have an average public school proficiency that does not differ significantly from 
that of Kentucky. The significance tests are based on a Bonferroni procedure for 
multiple comparisons that holds the probability of erroneously declaring the means of 
any two states to be different, when they are not, to no more than five percent. Two 
states — Idaho and Michigan — did not meet minimum school participation guidelines 
for public schools. Another jurisdiction — Washington, DC — withdrew from the Trial 
State Assessment after the data collection phase. Therefore, these three jurisdictions are 
not included in the comparisons depicted on the map on the following page. 
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